System for detecting the location of an endoscopic device during a medical procedure

ABSTRACT

A system for detecting the location of an endoscopic device inside a patients&#39; body comprises an endoscope assembly with an insertion tube having a set of codes indicative of an insertion depth and a rotational direction of the endoscopic device and a detector to detect the set of codes and calculate the location information. The images captured by the endoscopic device are automatically tagged with corresponding location information. Optionally, the endoscope assembly further comprises a system for generating three dimensional images and videos without increasing the number of cameras. Optionally, the endoscope assembly further comprises a system having a plurality of sensor devices for generating a real time image map of an endoscopic tip portion traversing a lumen. Optionally, the endoscope assembly further comprises a special garment for colonoscopy patients to preserve the modesty of the patients and to protect a physician from spraying fecal matter.

CROSS-REFERENCE

The present application relies on U.S. Patent Provisional ApplicationNo. 62/117,362, entitled “System and Method for Obtaining Real TimeImages of Endoscopic Procedures” and filed on Feb. 17, 2015, forpriority. In addition, the present application relies on U.S. PatentProvisional Application No. 62/162,078, entitled “Endoscope withInsertion Depth and Direction Detection System” and filed on May 15,2015, for priority.

The present application relates to U.S. patent application Ser. No.13/655,120, entitled “Multi-Camera Endoscope” and filed on Oct. 18,2012; U.S. patent application Ser. No. 13/212,627, entitled“Multi-Viewing Element Endoscope” and filed on Aug. 18, 2011; and U.S.patent application Ser. No. 13/190,968, entitled “Multi-CameraEndoscope” and filed on Jul. 26, 2011, all of which arecontinuation-in-part applications of U.S. patent application Ser. No.13/119,032, entitled “Multi-Camera Endoscope” and filed on Jul. 15,2011, which is a 371 National Stage Entry of PCT Application NumberPCT/IL2010/000476, of the same title and filed on Jun. 16, 2010, which,in turn, relies upon U.S. Provisional Patent Application No. 61/218,085,for priority.

The present application also relates to U.S. patent application Ser. No.14/505,387, entitled “Endoscope with Integrated Sensors”, and filed onOct. 2, 2014, and U.S. patent application Ser. No. 14/505,389, entitled“Endoscope with Integrated Sensors”, and filed on Oct. 2, 2014, which,in turn, rely upon the following applications:

U.S. Provisional Patent Application No. 61/886,572, entitled “Endoscopewith Integrated Location Determination”, and filed on Oct. 3, 2013;

U.S. Provisional Patent Application No. 61/890,881, entitled “Endoscopewith Integrated Pressure Sensing”, and filed on Oct. 15, 2013; and

U.S. Provisional Patent Application No. 61/980,682, entitled “System andMethod for Monitoring the Position of a Bending Section of AnEndoscope”, and filed on Apr. 17, 2014.

The present application also relates to U.S. Provisional PatentApplication No. 61/987,021, entitled “Real-Time Meta Tagging of ImagesGenerated by A Multiple Viewing Element Endoscope”, and filed on May 1,2014.

The present application also relates to U.S. patent application Ser.14/469,481, entitled “Circuit Board Assembly of A Multiple ViewingElements Endoscope”, filed on Aug. 26, 2014, which is hereinincorporated by reference in its entirety along with all of the priorityand related applications mentioned therein.

All of the above-mentioned applications and any priority applicationsmentioned therein are herein incorporated by reference in theirentirety.

FIELD

The present specification generally relates to systems for detecting thelocation of an endoscopic device inside a patient's body and inparticular to a system for automatic tagging of images captured byendoscopic devices with such location information. In embodiments, thepresent specification also relates to a system and method for generatinga real-time image map of the endoscopic device as it traverses apatient's lumen.

BACKGROUND

Medical probes such as endoscopes are used for examining and treatinginternal body structures such as the alimentary canals, airways, thegastrointestinal system, and other organ systems. Endoscopes haveattained great acceptance within the medical community since theyprovide a means for performing procedures with minimal patient traumawhile enabling the physician to view the internal anatomy of thepatient. Over the years, numerous endoscopes have been developed andcategorized according to specific applications, such as cystoscopy,colonoscopy, laparoscopy, upper gastrointestinal (GI) endoscopy andothers. Endoscopes may be inserted into the body's natural orifices orthrough an incision in the skin.

An endoscope is usually an elongated tubular shaft, rigid or flexible,having a video camera or a fiber optic lens assembly at its distal end.The shaft is connected to a handle, which sometimes includes an ocularlens or eyepiece for direct viewing. Viewing is also usually possiblevia an external screen. Various surgical tools may be inserted through aworking channel in the endoscope for performing different surgicalprocedures.

Endoscopes typically have a front camera, and may also additionallyinclude one or more side cameras, for viewing the internal organs, suchas the colon, and an illuminator for illuminating the field of view ofthe camera(s). The camera(s) and illuminators are located in a tip ofthe endoscope and are used to capture images of the internal walls ofthe body cavity being scanned. The captured images are sent to a maincontrol unit coupled with the endoscope via one of the channels presentin the tube, for display on a screen coupled with the control unit.

In an endoscopy system, the main control unit, which is used to processdata from an endoscope, is generally a separate unit while the endoscopeitself is a device that can be attached to the main control unit. Themain control unit comprises a front panel and/or a display screen fordisplaying operational information with respect to an endoscopyprocedure when the endoscope is in use. The display screen may beconfigured to display images and/or video streams received from theviewing elements of the multiple viewing elements endoscope.

During an endoscopic procedure, it is important to record the locationof findings or abnormalities noticed during the procedure. The locationof any pathological structure, such as a polyp, inside the body can bedefined by parameters such as insertion depth and the direction of theendoscope. None of the endoscopy systems currently available in themarket provide a convenient method of estimating the insertion depth anddirection of the device, with detailed position information, during aprocedure. Position coordinates are generally provided by a physicianbased on a rough estimate, which are then manually input into the systemto tag a specific image. In some conventional endoscopes, insertiontubes are marked with numbers which specify the depth of insertion atspecified intervals from the tip section. The physicians are required toestimate the insertion depth from these numbers and manually input itinto the system to tag images. However, this method is not accurate orconvenient due to the wide spacing between marks and the need tomanually input data.

Further, these endoscopy systems do not provide any means to accuratelyand/or automatically record the direction or rotational angle of anendoscope. There are endoscopy systems known in the art which usemagnetic fields to estimate the location of the scope within the body;however, such devices are costly and cumbersome to use as they requireexternal antennas to be set up for communication with the correspondingunit comprised in the distal tip of the scope. For example, ScopeGuide®by Olympus Corporation comprises electromagnetic coils emitting magneticfields which are detected by antennae and triangulated to create a 3Dimage of the endoscope while inside a patient's body. In U.S. Pat. No.6,610,007, Belson discloses an endoscope with “a selectively steerabledistal portion and an automatically controlled proximal portion. Theendoscope body is inserted into a patient and the selectively steerabledistal portion is used to select a desired path within the patient'sbody. When the endoscope body is advanced, an electronic motioncontroller operates the automatically controlled proximal portion toassume the selected curve of the selectively steerable distal portion.”

Conventional endoscopes also suffer from the drawback of having alimited field of view. The field of view is limited by the narrowinternal geometry of organs as well as the insertion port, which may beone of the body's natural orifices or an incision in the skin. Further,in order to know the exact position/orientation of an endoscope tipwithin a body cavity, an operating physician has to usually rely onexperience and intuition. The physician may sometimes become disorientedwith respect to the location of the endoscope's tip, causing certainregions of the body cavity to be scanned more than once, and certainother regions to not be scanned at all. For the early detection and cureof many diseases, such as cancer, it is essential that the body cavitybe examined in a manner such that no region remains un-scanned. Also,the precision of disease detection depends upon a thorough analysis ofthe images of the internal regions of the body cavity collected duringmultiple scans separated in time.

Further, with existing systems and methods, it is difficult andexpensive to provide three dimensional imaging or video recordingcapabilities in an endoscopic device. To enable three dimensionalimaging, a higher number of cameras and hardware, such as motionsensors, are required which significantly increases the size and cost ofendoscopic devices. At the same time, there is an ever-increasingrequirement to reduce the size of endoscopic devices to make theprocedure convenient for patients. Therefore, it becomes difficult forequipment manufacturers to provide three dimensional imaging or realtime video capabilities in the device.

Thus, what is needed is a method for measuring and recording locationinformation (both depth and direction) during an endoscopic procedure toenable automatic tagging of images with this data. In addition, what isneeded is a facile and inexpensive method to implement three-dimensionalimaging capabilities in endoscopic devices. What is also needed is amethod for providing real-time display of the device positioning withinthe body to enable better navigation capabilities.

What is also needed is a method for monitoring the complete movement ofa device within the body and notifying physicians in case a deviation isdetected from the normally expected movement. In addition, what is alsoneeded is a method to monitor the speed, velocity and acceleration ofthe device inside the body.

There is also a need for a method enabling an operating physician toscan a body cavity efficiently ensuring complete and uniform coverage.Further, there is also a need for a system and method of connecting anendoscope with a plurality of devices at various geographicallyseparated locations in order to enable sharing of endoscopic images inreal-time for obtaining improved treatment options for a patient.

Many patients are not comfortable undergoing an endoscopic procedure,and in particular a colonoscopy, as private body parts are exposed tothe medical staff during the procedure. Because of this reason, a largenumber of patients try to avoid colonoscopy examination for fear ofembarrassment or violation of religious practices, leading to anincreased risk of colon cancer in the population.

Thus, what is needed is an endoscopic system that can protect theprivacy of patients during a procedure as it would lead to a significantnumber of people opting for a colonoscopy investigation who would haveotherwise refrained from the same.

SUMMARY

The following embodiments and aspects thereof are described andillustrated in conjunction with systems, tools and methods, which aremeant to be exemplary and illustrative, not limiting in scope.

The present specification discloses an endoscope assembly comprising: anendoscope comprising: an insertion tube having a length extending from afront panel of said endoscope to a handle of said endoscope andcomprising on a surface throughout said length a plurality of codesindicative of insertion depth and direction of the endoscope when saidinsertion tube is inserted into a patient's body; a control handlecoupled to said insertion tube and configured to control variousfunctionalities of the endoscope; a detector configured to detect saidplurality of codes and transmit data representative of said codes to aprocessor; and, a main control unit coupled to said control handle andcomprising said processor, wherein said processor is adapted to estimatethe insertion depth and direction of the endoscope based on saidtransmitted data.

Optionally, the processor estimates the insertion depth as the distancebetween said front panel of the endoscope and an external referencepoint. The external reference point may comprise the position of apatient's anus.

Optionally, the processor estimates the direction of the endoscope asangular rotation with respect to a predefined reference direction. Thereference direction may extend in a perpendicular direction away from apatient's dorsal surface.

Optionally, each of said plurality of codes represents a unique set ofvalues of insertion depth and rotational angle of the endoscope.

The plurality of codes may comprise at least one barcode or numbersrepresenting the insertion depth at any position over the length of theinsertion tube.

Optionally, the detector comprises a hollow cylindrical structureadapted to receive the insertion tube through the hollow portion of thedetector and into the patient's body. The hollow cylindrical structuremay comprise two hollow semi-circular cylindrical portions which arejoined together to form the complete detector.

Optionally, the detector comprises a plurality of optical devices tocapture the image of codes marked on the section of the insertion tubepassing through the field of view of said optical devices. The opticaldevices may be embedded in the inner surface of a hollow cylindricalstructure.

Optionally, images captured by said endoscopic device are tagged with atimestamp wherein the timestamp is used to associate an image with aninsertion depth and direction of an endoscope.

Optionally, the endoscope further comprises a tip section adapted tocapture images and said processor is further adapted to receive datarepresentative of said images, wherein data of multiple images capturedby said tip section is merged with insertion depth and directioninformation to generate three dimensional images or videos.

Optionally, the endoscope assembly further comprises a garment to beworn by the patient for colonoscopy procedures and configured such thata rear section of the garment is re-positionable to expose an openingfor performing a colonoscopy procedure wherein said garment isconfigured to provide support for attaching and securing the detector.The rear section may comprise at least one of covers, flaps, Velcrostraps, buckles, and zippers configured to re-position the rear sectionand expose said opening. The garment may comprise underwear.

Optionally, the processor is further adapted to calculate theacceleration and speed of a distal tip of the endoscope inside thepatient's body based on the incremental change in values of insertiondepth and direction of the endoscope. The processor may be adapted tocompare the acceleration or speed of said distal tip with predefinedstandard parameters to detect inadequate screening due to incompleterotation or fast movement of the endoscope. The processor may be furtheradapted to activate an alert to notify the physician when inadequatescreening is detected.

Optionally, the detector comprises a hollow truncated conical structureadapted to receive the insertion tube through the hollow portion of thedetector and into the patient's body.

Optionally, the detector comprises a ring shaped base unit and at leastone vertical structure coupled to said base unit wherein said verticalstructure comprises a plurality of optical devices.

Optionally, the endoscope assembly is used to estimate the actual sizeof a pathological structure in the body by combining two separate imagesof said pathological structure captured from two insertions whose depthdifference is known.

The present specification also discloses a garment designed for apatient having a colonoscopy procedure comprising a rear section whichis re-positionable to expose an opening for performing the colonoscopyprocedure while simultaneously protecting the modesty of the patient bypreventing complete exposure of the patient's private body parts. Thegarment may comprise underwear. The garment may be configured to providesupport for firmly attaching a detector to the garment wherein saiddetector is configured to detect markings along the length of an outersurface of an insertion tube of the colonoscope, further wherein saidmarkings are indicative of insertion depth and direction of thecolonoscope.

The present specification also discloses an endoscope comprising atleast a first sensor on a distal end of a tip portion for capturing andtransmitting location information of the distal tip within a lumenduring an endoscopic procedure, the location information being processedto obtain a real time endoscopic image map of the tip portion traversingthe lumen.

Optionally, the endoscope further comprises a second sensor provided ata location outside the lumen for receiving the location information ofthe distal tip within the lumen during the endoscopic procedure.

The endoscopic image map may be displayed on one or more display devicescoupled with the endoscope for providing a real time location of theendoscope tip within the lumen.

Optionally, the first sensor transmits the location information of thedistal tip within the lumen by using wireless signals in thesub-gigahertz frequency field.

The first and second sensors may be any one of an accelerometer, a gyrosensor, and a radar based sensor.

The second sensor may be placed at a pre-defined location on the body ofa patient undergoing the endoscopic procedure, the location being inclose proximity to the point of insertion of the endoscope into thelumen, or may be placed at a pre-defined location on the bed of thepatient undergoing the endoscopic procedure.

Optionally, the second sensor provides a reference plane for obtaining areal time position of the distal end of the tip of the endoscope and apre-defined algorithm is used to process the reference plane and thelocation information to provide the real time endoscopic image map ofthe tip portion traversing the lumen.

The present specification also discloses a method of obtaining a realtime image map of an endoscope tip scanning a lumen during an endoscopicprocedure, the method comprising the steps of: recording an initialposition of the endoscope tip within the lumen as zero coordinates;obtaining a current position of the endoscope tip within the lumen;constructing a real time image map of the endoscope tip scanning thelumen by mapping the current position of the tip with reference to thezero coordinates; and displaying the image map on one or more displaydevices in real time.

Optionally, the method further comprises the step of storing the realtime images in a database, the stored images being used to performanalytical operations using images obtained from one or more previouslyconducted endoscopic procedures and a pre-defined algorithm.

Optionally, the method further comprises the step of marking one or moreregions of interest on the displayed images, the regions of interestbeing related to observation of an abnormality, the marking comprising acomment describing the abnormality.

Optionally, the method further comprises the steps of commencing arecording session when the endoscope tip is inserted into the lumen andstopping the recording session when the endoscope tip is extracted fromthe lumen.

The present specification also discloses an endoscope comprising: a tiphaving a distal end and a proximal end, the distal end being insertedinto a body cavity during an endoscopic procedure, and a navigationsensor coupled with the distal tip for capturing and transmitting alocation information of the distal tip within the body cavity; a handleportion coupled with the proximal end of the tip; and a main controlunit receiving the location information of the distal tip within thebody cavity, the main control unit being coupled with the handleportion, the main control unit comprising a wireless module connectingthe main control unit to the Internet for transmitting the locationinformation to one or more devices.

Optionally, a second sensor is coupled with the handle portion forreceiving the location information and transmitting the same to the maincontrol unit.

The wireless module may establish a link over one or more securedInternet standards for transmitting live video of endoscopic proceduresfrom the main control unit to one or more display devices. The wirelessmodule may be used for transmitting live video of endoscopic proceduresfrom the main control unit to one or more cellular phones present withina pre-defined range, the cellular phones having a pre-definedapplication installed therein, the cellular phones being used forbroadcasting the endoscopic procedures and comparing a currentendoscopic procedure with one or more previously conducted proceduresstored in a database via the installed application.

The present specification also discloses an endoscope comprising: afirst sensor on a distal end of a tip portion of the endoscope forcapturing and transmitting location information of the distal tip withina lumen during; and a second sensor in communication with the firstsensor, the second sensor provided at a location outside the lumen forreceiving the location information during an endoscopic procedure.

Optionally, the endoscope further comprises at least one wireless modulein communication with at least one of the first sensor and the secondsensor for wirelessly transmitting location information.

The aforementioned and other embodiments of the present specificationshall be described in greater depth in the drawings and detaileddescription provided below.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features and advantages of the present specificationwill be appreciated, as they become better understood by reference tothe following detailed description when considered in connection withthe accompanying drawings, wherein:

FIG. 1A illustrates a multiple viewing element endoscopy system inaccordance with an embodiment of the present specification;

FIG. 1B illustrates a perspective view of a tip section of a multipleviewing element endoscope in accordance with an embodiment of thepresent specification;

FIG. 2A illustrates a portion of an insertion tube of an endoscopicdevice in accordance with an embodiment of the present specification;

FIG. 2B illustrates a portion of an insertion tube comprising twodimensional barcode marks in accordance with an embodiment of thepresent specification;

FIG. 2C illustrates a portion of an insertion tube comprising threedimensional barcode marks in accordance with an embodiment of thepresent specification;

FIG. 2D is a flow chart listing the steps involved in a method of usingan endoscopic device with coded markings and a detector/reading unit todetermine position, in accordance with one embodiment of the presentspecification;

FIG. 3A depicts a vertical arrow in the direction of a patient'stailbone, showing an external reference point;

FIG. 3B is an illustration of a patient lying on his right side with aninsertion tube of an endoscopic device inserted into the patient's anus;

FIG. 4 is a snapshot of a number-based marking/coding scheme captured ata specific penetration depth of an endoscopic device, in accordance withone embodiment of the present specification;

FIG. 5A illustrates an exemplary undergarment used in conjunction withcolonoscopy procedures in accordance with an embodiment of the presentspecification;

FIG. 5B is a flow chart listing the steps involved in various methods ofsecuring a detector/reading unit to an undergarment and/or patient body,in accordance with some embodiments of the present specification;

FIG. 6A illustrates a colonoscopy procedure performed on a patient usingan endoscopic assembly in accordance with an embodiment of the presentspecification;

FIG. 6B illustrates a gastroscopy procedure performed on a patient usingan endoscope assembly in accordance with an embodiment of the presentspecification;

FIG. 7 illustrates an endoscope assembly comprising an endoscope, adetector/reading unit, and a specially designed undergarment, inaccordance with an embodiment of the present specification;

FIG. 8 illustrates an alternative embodiment of a detector/reading unitin the form of a truncated hollow conical structure, in accordance withone embodiment of the present specification;

FIG. 9 illustrates an alternate embodiment of a detector/reading unitcomprising a ring shaped base unit;

FIG. 10 illustrates a set of distances used to calculate the size of apathological structure viewed by an endoscopy system in accordance withone embodiment of the present specification;

FIG. 11 illustrates a physician and patient and an endoscopy system inaccordance with one embodiment of the present specification;

FIG. 12 is a flowchart illustrating a method of obtaining a real timeimage map of an endoscope tip from within a body cavity during anendoscopic procedure, in accordance with an embodiment of the presentspecification; and

FIG. 13 is a block diagram illustrating an exemplary video processingarchitecture, according to an embodiment of the present specification.

DETAILED DESCRIPTION

The present specification is directed towards multiple embodiments. Thefollowing disclosure is provided in order to enable a person havingordinary skill in the art to practice the invention. Language used inthis specification should not be interpreted as a general disavowal ofany one specific embodiment or used to limit the claims beyond themeaning of the terms used therein. The general principles defined hereinmay be applied to other embodiments and applications without departingfrom the spirit and scope of the invention. Also, the terminology andphraseology used is for the purpose of describing exemplary embodimentsand should not be considered limiting. Thus, the present invention is tobe accorded the widest scope encompassing numerous alternatives,modifications and equivalents consistent with the principles andfeatures disclosed. For purpose of clarity, details relating totechnical material that is known in the technical fields related to theinvention have not been described in detail so as not to unnecessarilyobscure the present invention.

It is noted that the term “endoscope” as mentioned to herein may referparticularly to a colonoscope or gastroscope, according to someembodiments, but is not limited only to colonoscopes or gastroscopes.The term “endoscope” may refer to any instrument used to examine theinterior of a hollow organ or cavity of the body.

The capability for recording the exact depth and direction of anendoscopic device inside a human body is imperative for tagging thelocation of any pathological structure, such as a polyp or any otherkind of abnormality, inside the body. Conventional endoscopy systems donot automatically record the location or insertion depth of theendoscopic device inside a patient's body. In some conventionalendoscopes, insertion tubes are marked with numbers which specify thedepth of insertion at specified intervals from the tip section.

In addition, current endoscopy systems do not provide any means torecord the direction or rotational angle of an endoscope. Also, currentendoscopy systems are not equipped to monitor the acceleration of thedevice inside the body and hence cannot detect and indicate occurrenceswhen there is inadequate screening due to incomplete rotation or fastmovement of the device.

The present specification describes an endoscopy system having thecapability to automatically detect the location of a device inside apatient's body and tag images captured by the viewing elements of theendoscopic device with this information. In an embodiment, the insertiondepth and rotational angle of the section of the endoscopic devicepresent inside the body are recorded in real time. In an embodiment, theinsertion tube of the endoscopic device is marked with a set of codesthat aid in establishing the insertion depth of the device with respectto a reference point and the rotational angle of the device with respectto a reference direction. The endoscopy system also comprises a detectoror a reading device to detect/read these codes from the insertion tubeas the device is lowered into a patient's lumen to establish the depthand direction of the endoscope with respect to reference parameters,such as a reference point or a reference direction. In an embodiment, areference point for measuring the insertion depth of an endoscope is afront panel of the tip section of the endoscope. In an embodiment, areference direction for measuring the direction of an endoscope is avector pointing in the direction of the tailbone of the patient. In someembodiments, the detector comprises cameras or optical readers to readthe codes marked on the insertion tube.

In an embodiment, the present specification provides a method ofobtaining real time position of an endoscope tip within a body cavityduring an endoscopic scan procedure. Embodiments of the presentspecification enable an operating physician to view a current positionof the endoscope tip within the lumen being scanned, on a display devicecoupled with the endoscope.

One of ordinary skill in the art would appreciate that there may bemultiple ways to develop the coding system for marking the insertiontube to ascertain the location of an endoscopic device without departingfrom the scope and spirit of the present specification. In anembodiment, an insertion tube is marked with barcodes along its lengthand circumference wherein each barcode comprises a unique set of valuesindicative of the insertion depth and rotation angle corresponding tothe area marked by said barcode. In this embodiment, the detector orreading unit comprises a barcode reader. One of ordinary skill in theart would appreciate that various types of barcodes could be used in theabove embodiment. A barcode is an optical machine-readablerepresentation of data relating to the object to which it is attached.In various embodiments, liner barcodes comprising various combinationsof parallel lines and/or two dimensional barcodes, which comprisegeometric patterns such as rectangles and dots, are marked on aninsertion tube for readying by a detector.

In some embodiments of the present specification, instead of the codesmarked over the outer surface of the endoscopic device, a plurality ofpassive RF-ID (Radio Frequency Integrated Devices) are positioned underthe outer layers of the endoscope insertion tube. In this case each RFIDcorresponds to a unique set of values of the insertion depth androtational angle of the endoscopic device in the patient body. Adetector/reading unit in the form of a RF reader is used to scan theRFIDs distributed under the outer layer of the endoscope to detect thelocation information of an endoscopic device as it is inserted into abody lumen.

In an embodiment, the detector/reading unit is a discrete unit from therest of the endoscopy system and is coupled to the patient's body todetect the codes marked on the insertion tube and establish theinsertion depth and rotation angle of the endoscope. One of ordinaryskill in the art would appreciate that there may be multiple ways toposition the detection device. In an embodiment, for example, in acolonoscopy procedure, the detector/reading unit is attached or fixed toa garment worn by the patient. In another embodiment, thedetector/reading unit is positioned in a special holder or housing toread the corresponding codes on the insertion tube. In an embodiment,the holder or housing containing the detector/reading unit is firmlycoupled or attached to the patient. In another example, in a gastroscopyprocedure, the detector/reading device is attached to a speciallydesigned mouth piece. In an embodiment, mouth pieces commonly used ingastroscopy procedures are configured to incorporate features such thata reading/detector unit can be firmly coupled to the mouth piece.

In an embodiment, for colonoscopy procedures, the present specificationdiscloses a garment, wearable by a patient and configured to securelyhold a detector/reading unit, wherein the detector/reading unitdetects/reads codes provided on an insertion tube of an endoscope as theendoscope is inserted into the anus of the patient. In an embodiment,the patient garment comprises underwear to which a detector/readingunit, such as an optical reader, is attached.

The use of a specific garment to hold the detector/reading unit providesan added benefit of maintaining the modesty/privacy of patientsundergoing a colonoscopy procedure. A large number of patients try toavoid colonoscopy investigation for fear of embarrassment or violationof religious practices as they have to expose their private body partsto the medical staff. In an embodiment, the patient garment described inthe present specification prevents the complete exposure of thepatient's private body parts, thereby maintaining his or her modesty toa certain extent.

In the above mentioned embodiments, a detector/reading unit and anendoscopic device are initially aligned to a specific reference pointand a reference direction. All changes in the position of the endoscopicdevice are measured relative to these reference parameters to establishany incremental change in depth or direction.

In an embodiment, the system described in the present specificationprovides an inexpensive and efficient method of creatingthree-dimensional (3D) still images or live videos without increasingthe size of a distal tip of an endoscope. Usually, a higher number ofcameras and motion sensors is required to create 3D images or videowhich significantly increases the size and cost of endoscope devices.The present specification provides an inexpensive method of creating 3Dstill images or live videos by fusing or merging 2D image data recordedat varying levels of depth and rotational angles.

In an embodiment, the present specification also provides a method forestimating the size and location of pathological structures, such aspolyps.

In an embodiment, the present specification enables easier navigationfor physicians who are not adept in navigating an endoscopic devicewithout additional information on the depth and rotational angle. In anembodiment, the present specification provides means for displaying theinsertion depth and rotational angle of an endoscope along with an imageof the distal tip inside a patient's lumen in real time.

In another embodiment, with the aid of depth and direction data, asystem processor monitors the acceleration of an endoscopic deviceinside a patient's body and issues automatic alerts, notifications orwarnings. The speed at which the endoscopic device is moving through thepatient's body is measured by calculating the difference, both ininsertion depth and in timing of detection, between two consecutivereadings of markings on the endoscopic device by the processor.Specifically, a processor receives data indicative of a time stampedinsertion depth from an optical reading device. Periodically, theprocessor calculates a speed of insertion by determining a firstinsertion depth and its first associated time, determining a secondinsertion depth and its second associated time, obtaining a differencebetween the second insertion depth and first insertion depth, obtaininga difference between the first associated time and second associatedtime, and dividing the two differences.

In an embodiment, notifications are issued if an endoscopic device ispushed or pulled too fast for reliable detection of pathologicalstructures or for safe traverse of a body lumen. Specifically, athreshold speed value is stored in a memory and compared, on a real-timebasis, to the speed determined as described above. When the calculatedspeed exceeds the threshold speed, as determined by the processor, theprocessor issues a notification in the form of a signal to a vibrator inthe endoscope handle, a signal to an audio speaker in the display, or asignal to the display to visually present an icon, image, color, orother picture. In another embodiment, automatic notifications are issuedif a device is not rotated fully to cover an entire 360 degrees with theside cameras of an endoscope. In embodiments of the presentspecification, the optimal speed of the movement of a tip section forscanning a specific section of a lumen is pre-determined and input intothe endoscopic system. In an alternate embodiment, the endoscopic systemis intelligent and determines the optimal speed for scanning varioussections of the lumen based on historical data. In an embodiment, thehistorical data comprises the scanning speed used for scanning a sectionof lumen during various procedures conducted in the past. In case thereis significant deviation from these optimal reference speeds,appropriate notifications are issued to the physician.

In an embodiment, the present specification discloses a novel endoscopysystem comprising a plurality of sensors for generating a real timeimage map of the endoscopic tip portion traversing a body lumen. In anembodiment, a first sensor device is deployed on the tip section of theendoscope for capturing and transmitting location information of thedistal tip within a lumen and a second sensor device is deployed at alocation outside the lumen for receiving the location information of thedistal tip within the lumen during a procedure. Embodiments of systemsand methods enable mapping images captured by the endoscope from withinthe lumen in real time and placing the captured images in a referenceframe. A method of the present specification enables an operatingphysician to perform a complete endoscopic scan of a body cavity duringa single scanning session, without missing any region therein.Embodiments of the specification also provide a method of transmittingdata from within a body lumen in real time, thereby providing anaccurate location of an endoscope tip within the lumen on a displaydevice coupled with the endoscope. The coordinates of the endoscope tipwithin the lumen are recorded in real time, thereby providing a map ofthe areas of the lumen scanned by the endoscope. Embodiments of thespecification also provide a method of marking regions of interestwithin the captured images mapped onto the pre-defined reference frameand recording one or more of the physician's findings corresponding tothe marked regions. The markings enable comparisons between the markedregions across images captured at different times/dates. Thus, variousembodiments of the specification enable early detection of diseases.

The present specification also provides a method of wirelesslytransmitting endoscope scan data from a main control unit of theendoscope to one or more devices, including mobile devices and displaydevices. The present specification also provides a method for livestreaming of an endoscopic scanning procedure of a body cavity onmultiple devices, such as mobile devices and display devices.

Reference is now made to FIG. 1A, which shows a multiple viewingelements endoscopy system 100 in which the insertion depth and directiondetection system, as well as the real time image mapping method, of thepresent specification can be implemented. In an embodiment, the system100 includes a multiple viewing elements endoscope 102. In anembodiment, the multiple viewing elements endoscope 102 includes ahandle 104, from which an elongated shaft 106 emerges. Elongated shaft106 terminates with a tip section 108 which is turnable by way of abending section 110. Handle 104 is used for maneuvering elongated shaft106 within a body cavity. The handle 104 includes one or more buttonsand/or knobs and/or switches 105 which control bending section 110 aswell as functions such as fluid injection and suction. Handle 104further includes at least one working channel opening 112 through whichsurgical tools may be inserted.

A utility cable 114, also referred to as an umbilical tube, connectsbetween handle 104 and a main control unit 199. Utility cable 114includes therein one or more fluid channels and one or more electricalchannels. The electrical channel(s) includes at least one data cable forreceiving video signals from front and side-pointing viewing elements ofthe endoscope 102, as well as at least one power cable for providingelectrical power to the viewing elements and to discrete illuminators.

The main control unit 199 contains the controls required for displayingthe images of internal organs captured by the endoscope 102. In anembodiment, the main control unit 199 governs power transmission to theendoscope's 102 tip section 108, such as for the tip section's viewingelements and illuminators. In an embodiment, the main control unit 199further controls one or more fluid, liquid and/or suction pump(s) whichsupply corresponding functionalities to the endoscope 102. Inembodiments, one or more input devices 118, such as a keyboard, a touchscreen and the like are connected to the main control unit 199 for thepurpose of human interaction with the main control unit 199. In theembodiment shown in FIG. 1A, the main control unit 199 is connected to ascreen/display 120 for displaying operation information concerning anendoscopy procedure when the endoscope 102 is in use. The screen 120 isconfigured to display images and/or video streams received from theviewing elements of the multiple viewing elements endoscope 102. Thescreen 120 may further be operative to display a user interface forallowing a human operator to set various features of the endoscopysystem 100.

Optionally, the video streams received from the different viewingelements of the multiple viewing elements endoscope 102 are displayedseparately on at least one monitor/screen 120 by uploading informationfrom the main control unit 199, either side-by-side or interchangeably(namely, the operator may switch between views from the differentviewing elements manually). Alternatively, these video streams areprocessed by the main control unit 199 to combine them into a single,panoramic video frame, based on an overlap between fields of view of theviewing elements. In an embodiment, two or more displays are connectedto the main control unit 199, each for displaying a video stream from adifferent viewing element of the multiple viewing elements endoscope102.

Referring to FIG. 1B, a perspective view of a tip section 150 of amultiple viewing elements endoscope is shown. In an embodiment, the tipsection 150 includes therein a front-pointing viewing element 154 whichcaptures images through a hole in a distal surface 156 of tip section150.

At least one discrete front illuminator 158, which is, in an embodiment,a light-emitting diode (LED), is associated with front-pointing viewingelement 154 and is used for illuminating its field of view throughanother hole in distal end surface 156. In embodiments of the presentspecification, the LED comprises a white light LED or an infrared lightLED or a near infrared light LED or an ultraviolet light LED, or acombination of wavelengths. The term “discrete”, in regard to frontilluminator 158, may refer to an illumination source which generateslight internally—in contrast to a non-discrete illuminator which may be,for example, a fiber optic merely transmitting light generated remotely.In another embodiment, two or more discrete front illuminators 158 arepresent on distal surface 156 of tip section 150, for supplying overallstronger illumination and/or for increasing the angular coverage of theillumination. In an embodiment, these two or more discrete frontilluminators are located next to one another so that they share a sameprotective window on distal surface 156.

A front fluid injector 160 is used for cleaning at least one offront-pointing viewing element 154 and discrete front illuminator 158.In an embodiment, the distal end surface 156 includes a hole defining aworking channel 162, which may be a hollow tube configured for insertionof a surgical tool to operate on various tissues. A pathway fluidinjector 164, defined by another hole in distal end surface 156, is usedfor inflating and/or cleaning the body cavity into which endoscope tip150 is inserted.

In an embodiment, the tip section 150 further includes therein aside-pointing viewing element 166 which captures images through a holein a cylindrical surface 155 of the tip section. At least one discreteside illuminator 172, which is optionally similar to discrete frontilluminator 158, is associated with side-pointing viewing element 166and used for illuminating its field of view through another hole incylindrical surface 155. In an embodiment, a side fluid injector 170 isused for cleaning at least one of side-pointing viewing element 166 anddiscrete side illuminator 172.

In another embodiment, two or more discrete side illuminators 172 arepresent in the tip section 150, such as for supplying overall strongerillumination and/or for increasing the angular coverage of theillumination. In an embodiment, these two or more discrete sideilluminators are located next to one another so that they share a sameprotective window on the cylindrical surface of the tip section. Inorder to prevent tissue damage when cylindrical surface 155 of tipsection 150 contacts a side wall of the body cavity, side fluid injector170, side-pointing viewing element 166, and side illuminators 172 areoptionally located in a notch or depression 168 in cylindrical surface155. In this manner, side fluid injector 170 is elevated from depression168 but still may not significantly protrude from the level ofcylindrical surface 155. The elevation of side fluid injector 170enables it to inject fluid onto side-viewing element 166 and discreteside illuminators 172. In yet another embodiment, a side-viewingelement, one or more side illuminators and a side fluid injector are notlocated in a depression, but are at the same level as cylindricalsurface of the tip section. In various embodiments, another side-viewingelement, one or more side illuminators, and another side fluid injectorare positioned on an opposite side of surface 155 from side-viewingelement 166, side illuminators 172, and side fluid injector 170 toprovide an additional field of view.

Endoscopic Position Determination

FIG. 2A illustrates a portion of an insertion tube 200 of an endoscopicdevice in accordance with an embodiment of the present specification. Asshown in FIG. 2A, the insertion tube 200 comprises a distal section 201which includes a front panel 202 at its distal end. The front panel 202comprises a front viewing element 203. Front illuminators 204 a, 204 band 204 c illuminate the field of view of viewing element 203. A frontworking channel opening 206 allows the physicians to insert medicaltools into and through a working channel of the endoscope and performprocedures on target areas viewed through the front viewing element 203.In another embodiment, a jet channel opening 205 of a jet channel islocated on front panel 202. Jet channel opening 205 is configured toprovide high-pressure jets of fluid, such as water or saline, forcleaning the walls of a body cavity being viewed.

In an embodiment, the insertion tube 200 comprises a plurality of codes207 marked over its external surface or cover. The insertion tube 200 isan elongate, tubular section of the endoscopic device having a length, aproximal end and a distal end and configured to be inserted into a lumenof a patient. In some embodiments, the proximal end 213 of the insertiontube is defined by a distal end of the endoscope handle (104 in FIG. 1A)and the distal end of the insertion tube is defined by the front panel202. In some embodiments, the plurality of codes 207 extends along theentire length and about the entire circumference of the insertion tube200, extending from the handle to the front panel 202. In otherembodiments, the plurality of codes 207 extends along only a portion ofthe length of the insertion tube 200. In some embodiments, each codewithin the plurality of codes 207 corresponds to a unique set oflongitudinal and rotational data relative to certain predefinedreference parameters. In an embodiment, this data is indicative of thelocation information of a specific point or area on the insertion tube200 associated with the position of the code on the tube 200. Thedetector/reading unit 208 is configured to detect/read the codes markedon the section of insertion tube 200 that passes through its field ofview 211 as the endoscopic device is inserted into the patient's body.In an embodiment, the reading unit 208 is in data communication with aprocessor 209 which is configured to receive the information on codes207 detected by detector/reading unit 208, decrypt this information toestimate the insertion depth and rotational angle and, in an embodiment,transmit the decrypted information further to the main control unit 210of the endoscope assembly. In some embodiments, the detector/readingunit 208 and processor 209 are a unified unit. In other words, in someembodiments, the processor 209 is housed within the detector/readingunit 208. In some embodiments, the processor 209 performs minimalprocessing and is mainly responsible for transferring raw readings tothe main control unit 210. In other embodiments, the processor 209 isresponsible for complete depth, rotation, speed, and accelerationanalysis and passes the resulting data to the main control unit 210. Instill other embodiments, the processor 209 is also responsible forsending video obtained by scanning cameras to the main control unit 210.

One of ordinary skill in the art can appreciate that there may bemultiple ways to develop the coding scheme and to define the referenceparameters. In some embodiments, the reference points for measuring thelongitudinal information or the insertion depth are the front panel 202and the detector/reading unit 208, wherein the distance between thefront panel 202 and the detector/reading unit 208 signifies theinsertion depth of the endoscopic device at any given position. In thisembodiment, each code within the plurality of codes 207 is indicative ofa relative distance from the front panel 202. The codes marked along thecircumference of the insertion tube 207 at a specific depth indicate thesame distance from the front panel 202.

In some other embodiments, the reference points for measuring thelongitudinal information or the insertion depth are the front panel 202and the position of a patient's anus, wherein the distance between thefront panel 202 and anus signifies the insertion depth of the endoscopicdevice at any given position. In this embodiment, the coding scheme isdevised such that each of the codes 207 indicate the distance betweenthe front panel 202 and the patient's anus which provides an estimate ofthe depth at which investigation is performed.

In some embodiments, the angular movement or rotation of the endoscopicdevice inside the body is measured with respect to the direction of someexternal reference point, such as a patient's tailbone. In variousembodiments of the present specification, angular movement of theendoscopic device is defined as the angle of rotation of the entireinsertion tube along its longitudinal axis. Angular movement occurs asthe physician turns his palm while manipulating the endoscopic device,causing the insertion tube to rotate about the longitudinal axis of thedevice.

Reference is made to FIG. 3A, which shows a person 300 with arrow 301extending in a cephalic direction (toward the person's head) along adorsal surface of the person with respect to the person's tailbone.Arrow 301 indicates the insertion direction of an endoscopic device asthe insertion tube is introduced into a patient's anus. FIG. 3B is anillustration of a patient 310 lying on his right side with an insertiontube 315 of an endoscopic device inserted into the patient's 310 anus.Arrow 316 illustrates a direction extending perpendicularly away from adorsal surface of the patient 310 and, in various embodiments, indicatesa 0° reference angle for angular movement of the insertion tube 315.Arrow 317 illustrates a direction extending perpendicularly away from aright side of the patient 310 and, in various embodiments, indicates a90° rotational angle in a clockwise direction, or 270° rotational anglein a counter-clockwise direction, for angular movement of the insertiontube 315 with respect to the 0° reference angle indicated by arrow 316.Arrow 318 illustrates a direction extending perpendicularly away from aventral surface of the patient 310 and, in various embodiments,indicates a 180° rotational angle for angular movement of the insertiontube 315 with respect to the 0° reference angle indicated by arrow 316.Arrow 319 illustrates a direction extending perpendicularly away from aleft side of the patient 310 and, in various embodiments, indicates a270° rotational angle in a clockwise direction, or 90° rotational anglein a counter-clockwise direction, for angular movement of the insertiontube 315 with respect to the 0° reference angle indicated by arrow 316.

In an embodiment, referring again to FIG. 2A, each of the codes from theplurality of codes 207 marked on the surface of insertion tube 200 isindicative of a specific angle of rotation of the tube area marked bythat code with respect to the direction of arrow 316 of FIG. 3B. As theendoscopic device is inserted and rotated inside a patient's body, thedetector/reading unit 208 detects the codes 207 marked on the section ofinsertion tube 200 passing through its field of view 211 and records thesame for further processing. For angular rotation, the codes 207 areplaced at predefined positions, starting with a first position definedas the zero rotation angle point. As the codes 207 move from that zerorotation angle point, they indicate relative rotational angles. Theoptical reader(s) of the detector/reading unit 208 records the codes 207at different locations, indicative of insertion depth and relativeposition around the periphery, and tracks the locations, thereby beingable to determine how many times, and in what direction, the endoscopeinsertion tube has moved. In embodiments, the system uses theinformation recorded by the detector unit 208 to calculate the exactangle of rotation of the endoscopic device with respect to the directionof arrow 316 in FIG. 3B.

In an embodiment, the system is configured such that, as the physiciancaptures any image data, the main control unit 210 of the endoscopeassembly maps the imaging data received from the viewing element 203with the location information received from the processor 209 and tagsthe image with this location information. In an embodiment, the imagedata comprises stationary two dimensional images. In another embodiment,the image data comprises three dimensional images. In an alternateembodiment, the image data comprises a video of an interior portion of abody lumen captured for a specific time duration or a real time video.In all the above embodiments, the system stamps the image data with thelocation information received from the detector/reading unit 208. Insome embodiments, the processor 209 is not present and the main controlunit directly receives the location information from the reader 208either through cables or through wireless communication.

In some embodiments, the plurality of codes 207 is etched on the surfaceof insertion tube 201 at the time of manufacturing of the endoscopeassembly. In another embodiment, an independent strip or cover/skincomprising codes 207 is inserted or pasted on the insertion tube 200either by the original equipment manufacturer or the user. The totallength or specific sections of insertion tube 200 on which the codes 207are marked may vary in different embodiments depending on the codingscheme. The exact nature and configuration of the detector/reading unit208 mentioned in the above embodiments depends on the type of codingschemes used on the endoscopic device.

In some embodiments, the plurality of codes 207 has a uniform patternacross the length of the endoscopic device. However, in such a codingsystem, only relative measurement is possible, i.e. each set of codeswithin the plurality of codes 207 can be used to determine the relativechange in position of the insertion device and not the insertion depthin absolute terms. To enable measurement of insertion depth in theabsolute terms, in some embodiments, such as the embodiment shown inFIG. 2A, a non-uniform plurality of codes 207 is used in which thecoding pattern varies across the length of the endoscopic device.

In an embodiment, the plurality of codes 207 comprises simple linearbarcodes and accordingly the detector/reading unit 208 comprises asimple optical device, such as a barcode reader. Reference is made toFIG. 2B showing an insertion tube 220 with a plurality of codes 227marked over the outer surface of tube 220. In this coding scheme, thewidth of vertical lines and the spacing between alternate lines are usedto define the longitudinal information and coding determined by thepatterns of the lines is used to define the rotational information atany point. In another embodiment illustrated in FIG. 2C, two dimensionalbarcodes are used which comprise layers of a specific graphical pattern247 marked over the outer surface of insertion tube 240. The encodingschemes comprising two dimensional barcodes provide a moreadvanced/detailed level of encoding that puts less load on theprocessor.

FIG. 2D is a flow chart listing the steps involved in a method of usingan endoscopic device with coded markings and a detector/reading unit todetermine position, in accordance with one embodiment of the presentspecification. At step 262, an endoscopic device having coded markingsalong its outer surface (or the outer surface of its insertion tube) isinserted through a detector/reading unit and into a body lumen of apatient such that said coded markings are read by detector/reading unit.Then, at step 264, data relating to the read markings is transmitted toa processor. The data is converted into positional information by theprocessor by matching the read coded markings with codes in a look-uptable at step 266. The positional information is saved into a database,in association with a timestamp, at step 268. At step 270, the timestampis matched to images to associate specific images with a specificposition in the lumen. Therefore, images captured by the endoscopicdevice are tagged with a timestamp and the timestamp is used toassociate the image with an insertion depth and direction of theendoscope.

In some embodiments of the present specification, such as theembodiments shown in FIGS. 2B and 2C, in addition to the coding systemas described above, the insertion tube also comprises a conventionalsystem of marking the insertion depth in numerical terms which may bephysically inspected by the physicians. The strips 222, 242 in FIG. 2Band FIG. 2C respectively, contain information on the insertion depth innumerical terms which may be physically inspected by the physicians.Using a dual system as described in FIGS. 2B and 2C provides analternative to the physicians in case they do not want to use theautomatic location detection mechanism described in the presentspecification. In some embodiments, a similar strip comprising theinformation on the angular movement of the device is included across thecircumference of the endoscopic device.

The exact resolution or the minimum unit of insertion depth and angularmovement that can be measured depends on the coding scheme used forencoding the location information. Certain coding schemes, such as thetwo dimensional coding scheme mentioned in FIG. 2C, are more advancedand provide a higher level of resolution.

In an embodiment of the present specification, conventional number basedmarkings are used both as codes to be automatically detected by a reader(for example, via computerized vision) and to enable backwardcompatibility by providing an option to the physicians to use a manual,visual inspection method for determining the insertion depth. Thenumbers represent the insertion depth at any position over the length ofthe insertion tube of the endoscope. A detector/reading unit, comprisingan optical device such as a camera, is incorporated into the endoscopysystem to capture images of the conventional numbers marked on thesection of the insertion tube passing through the field of view of thedetector/reading unit. In some embodiments, wherein the spacing betweenthe numbers or marks is wide, a higher number of optical devices arerequired.

In another embodiment of the present specification, an insertion tube ofan endoscopic device includes number based codes/marks with narrowspacing between two alternate marks to reduce the processing load on thesystem and enhance the reliability of the location data estimationprocess. In this embodiment, the number of optical devices in thedetector/reading unit is relatively low which reduces the system weight,cost and size. Reference is made to FIG. 4 which illustrates a numberbased sample marking/coding at a specific penetration depth. The ribbon400 depicted in FIG. 4 has been removed from an endoscope and spread outas a strip. In use, the ribbon 400 would be wrapped about a portion ofan endoscope insertion tube corresponding to an insertion depth of 35 cmsuch that the numbers ‘35’ could be read by a physician from all sidesof the insertion tube. The ribbon 400 comprises numbers 401 which depictthe insertion depth at the specified level of where the ribbon 400 wouldbe placed on the insertion tube. In addition, the coding systemcomprises special marks/codes 405, 406, 407, 408 which are detected bythe reader to determine the rotational angle 403 of the insertion tube.For example, in an embodiment, when a detector/reading unit has a fieldof view encompassing region 410, the detector/reading unit reads mark405 and the system processes this information as a rotational angle of67.5° (between 45° and 90°). When a detector/reading unit has a field ofview encompassing region 412, the detector/reading unit reads marks 406,407, 408 and the system processes this information as a rotational angleof 225°. In an embodiment, the physician can manually estimate theinsertion depth and rotational angle by reading the number based codesat any level and by knowing a reference angle indicated by markings 402.In another embodiment, the optical reader automatically scans the imagesof the codes/marks of the section of insertion tube that passes throughits field of view and uses this image information to decode the exactinsertion depth and rotational angle with. In various embodiments, anembedded or independent processing unit configured for decoding thecodes/marks calculates the exact insertion depth and rotational angle.

Positioning the Detector/Reading Unit in Colonoscopy Procedures

One of ordinary skill in the art would appreciate that there may bemultiple ways to position a detector/reading unit in the aboveembodiments without departing from the overall spirit and scope of thisspecification. In an embodiment, for example, in a colonoscopyprocedure, the detector/reading unit is attached or fixed to a garmentworn by the patient. In an embodiment, the garment comprises a specialtype of underwear designed for patients that protects the modesty ofpatients by limiting the areas of exposure during the procedure and atthe same time allows implementing the novel system of locationestimation disclosed in embodiments of the present specification. In anembodiment, the underwear comprises a section on the rear side which canbe partially opened up and is adapted to receive and hold adetector/reading unit, such as an optical reader, such that thedetector/reading unit is able to read the codes marked on an insertiontube as it is inserted into a patient' anus. Reference is made to FIG.5A, which illustrates an undergarment 501, such as an underwear,configured to hold a detector/reading unit, in accordance with anembodiment of the present specification. As shown in FIG. 5A, a person500 is shown wearing an undergarment 501. Section 502 depicts theportion of the undergarment 501 which is configured to be partiallyopened such that a physician can insert an endoscopic device into thepatient's anus. In an embodiment, the undergarment 501 is alsoconfigured to securely hold a detector/reading unit in place as anendoscopic device is passed through the reader and advanced into apatient's anus. In an embodiment, the undergarment 501 comprises flapsor Velcro straps located at the interface between the free ends ofsection 502 and the remainder of the undergarment 501 which are used tosecure section 502 when in a closed position. However, any other knownmethods of opening up and closing the portion comprising section 502 maybe used in various embodiments of the present specification. In anembodiment, the undergarment 501 is made of a material such that it canbe washed and disinfected after every use. In another embodiment, theundergarment 501 is fabricated from a disposable material and intendedfor single use.

The section 502 is large enough to enable a physician to convenientlymanipulate the patient's buttocks to ease the insertion of theendoscope. In an embodiment, the section 502 comprises a plurality offlaps which can be opened up and, once the insertion stage is completed,the physician repositions the flaps to conceal the patient's buttocks asmuch as possible. Moreover, the physician at this stage attaches orfastens the detector/reading unit such, as an optical reader, to thepatient's body or to the undergarment using straps, buckles, clamps orother similar fastening means. In some embodiments, the fastening meanscomprise part of the undergarment. In other embodiments, the fasteningmeans comprise part of the detector/reading unit. In still otherembodiments, complimentary fastening means are provided on both theundergarment and the detector/reading unit wherein said complimentaryfastening means couple with one another to secure the detector/readingunit to the undergarment and, by extension, to the patient's body.

FIG. 5B is a flow chart listing the steps involved in various methods ofsecuring a detector/reading unit to an undergarment and/or patient body,in accordance with some embodiments of the present specification. Atstep 512, a patient puts on a specially designed undergarment inaccordance with various embodiments of the present specification. Thepatient lies on a table on his side at step 514. Then, at step 516, arear compartment of the undergarment is opened by the physician bymoving flaps/covers on the rear side of the undergarment to expose aworking space providing access to the patient's anus. In one embodiment,at step 518, the physician first inserts the insertion tube of anendoscopic device through a detector/reading unit. The physician theninserts the insertion tube into the anus of the patient at step 520. Thedetector/reading unit is then attached to the undergarment or to thepatient at step 526. Optionally, the physician repositions theflaps/covers of the undergarment to preserve patient privacy at step528. In another embodiment, following step 516, the physician firstinserts the insertion tube of an endoscopic device into the anus of thepatient at step 522. The physician then closes a sectionaldetector/reading unit about the insertion tube at step 524. Thedetector/reading unit is then attached to the undergarment or to thepatient at step 526. Optionally, the physician repositions theflaps/covers of the undergarment to preserve patient privacy at step528.

FIG. 6A illustrates a colonoscopy procedure performed on a patient usingan endoscope assembly in accordance with an embodiment of the presentspecification. As shown in FIG. 6A, a patient 600 is shown wearing aspecially designed underwear 601 which is adapted to partially open fromthe rear side to enable conducting a colonoscopy procedure. In anembodiment, the underwear 601 comprises flaps 604 which are lifted orpositioned to open the rear side section of underwear 601 and enable theinsertion of insertion tube 602 into the patient's 600 anus. In anembodiment, a detector/reading unit 603, such as an optical reader, iscoupled to the patient body. In an embodiment, the underwear 601comprises means to provide firm support to the detector/reading unit603. In an embodiment, the detector/reading unit 603 is coupled to amain control unit 606 either wirelessly or through a wire 605 for powersupply and data link.

Positioning the Detector/Reading Unit in Gastroscopy Procedures

FIG. 6B illustrates a gastroscopy procedure performed on a patient 620using an endoscope assembly in accordance with an embodiment of thepresent specification. As shown in FIG. 6B, a patient 620 is shownwearing a mouthpiece 627 which is configured in a firm position aroundthe patient's 620 mouth with the help of a band 628. In an embodiment,the mouthpiece 627 is similar to any conventional mouthpiece used forgastroscopy procedures. In another embodiment, the mouthpiece 627 isspecially configured for the embodiments of the present specificationwherein the mouthpiece 627 comprises a means to couple with adetector/reading unit 623 and hold it in a firm position during theprocedure. In yet another embodiment, the mouthpiece 627 anddetector/reading unit 623 comprises one unified unit. The endoscope 630comprises a control handle 639 which is coupled to an insertion tube 632and a main control unit 636 through a cord 640. In an embodiment, thephysician inserts the insertion tube section 632 of the endoscopicdevice in the patient's 620 mouth through a central opening in thedetector/reading unit 623 and performs the medical procedure. Inembodiments, the detector/reading unit 623 comprises detection devicessuch as an optical reader(s) for detecting the location information ofthe device inside the patient's body and communicating the same to maincontrol unit 636. In an embodiment, the detector/reading unit 623 iscoupled to the main control unit 636 through a wire 640. In anotherembodiment, the detector/reading unit is coupled to the main controlunit thorough a wireless link. In some embodiments, the detector/readingunit comprises batteries to meet its power requirements.

FIG. 7 illustrates an endoscope assembly comprising an endoscope 709, adetector/reading unit 703, and a specially designed undergarment 701, inaccordance with an embodiment of the present specification. As shown inFIG. 7, a specially designed patient underwear 701 is configured suchthat it can be opened from a rear side 708 to expose a section throughwhich the colonoscopy procedure is to be performed. The rear portion 708of the underwear 701 is configured such that it can be opened by openingor repositioning rear side covers or flaps 704. The exact number offlaps or covers 704 may vary in various embodiments. One of ordinaryskill in the art would appreciate that there may be other alternativemeans to enable the partial opening of the rear side 708 of theunderwear 701 without departing from the spirit and scope of the presentspecification. In some embodiments, buttons, buckles or zippers are usedto close or open the rear side 708 of the underwear 701.

In some embodiments, the underwear 701 is adapted to provide support tohold the detector/reading unit 703 in a firm position. In someembodiments, the underwear 701 comprises additional structural elements706, such as struts, buckles or chains, to provide support to thedetector/reading unit 703. In an embodiment, the underwear 701 isconfigured such that the structural elements 706 comprise a housing toreceive the detector/reading unit 703 which is adapted to hold thedetector/reading unit 703 firmly in place.

As shown in FIG. 7, in an embodiment, the detector/reading unit 703 isstructured in a hollow cylindrical shape such that the hollow portion ofthe cylinder allows the physician to insert medical tools through it andinto the patient's body. In an embodiment, the inside portion of thecylindrical detector/reading unit 703 comprises a plurality of opticaldevices 710 for detecting the codes 707 marked on the endoscopic device709 and for estimating the location information, comprising theinsertion depth and rotational angle of the endoscopic device 709, basedon said codes 707. In embodiments, the optical windows that cover theoptical devices 710 are sealed and are resistant to water and otherliquids such as intestine fluids and most of all, sterilizers anddisinfectants. The exact relationship between the detector/reading unit703 and the optical devices 710 is dependent on the type ofmarkings/codes 707 and spacing between them.

In an embodiment, the detector/reading unit 703 comprises a monolithicstructure. In another embodiment, the detector/reading unit 703comprises a plurality of modular components which can be attachedtogether to configure the detector/reading unit 703. In an embodiment,the detector comprises two semi-circular cylindrical sections which canbe attached together with clamps. In an embodiment, the twosemi-circular cylindrical sections are individual components and can beseparated completely from one another. In another embodiment, the twosemi-circular cylindrical sections are joined by a hinge and can beopened and closed about said hinge. In an embodiment, the physicianfirst inserts the insertion tube 702 of the endoscopic device 709 in thepatient's body and subsequently the two halves of the detector/readingunit 703 are attached around the insertion tube 702.

In one embodiment, the detector/reading unit 703 is an optical reader.However, in other embodiments, the detector/reading unit 703 comprisesany other type of reading equipment, such as an RF reader. The exactconfiguration of the detector/reading unit 703 depends on the codingsystem used to encrypt the location information of the endoscopic device709.

In an embodiment, the insertion tube 702 of the endoscope assembly isencrypted with codes 707, such as barcodes, which comprise informationon the insertion depth and rotational angle of the endoscopic device 709with respect to certain reference parameters. As the distal section ofthe insertion tube 702 is inserted into the patient's anus through thedetector/reading unit 703, the optical devices 710 scan the codes 707marked on the section of insertion tube 702 which are in the field ofview of the optical devices 710. The optical devices 710 communicate thecode 707 information to a processing unit which, based on specificalgorithms, decrypts the location information comprising the insertiondepth and rotational angle of the endoscopic device 709. In variousembodiments, the processing unit transmits the information to a maincontrol unit through a cable 705. In an embodiment, the cable 705provides the power supply to the detector/reading unit 703. In anotherembodiment, the cable 705 provides a two way data link for configuringthe detector/reading unit 703 and for transmitting the real timelocation data from the detector/reading unit 703 to the main controlunit. In another embodiment, the processing function is performed by aprocessor located in the main control unit and the detector/reading unit703 directly transmits the captured code 707 information to the maincontrol unit. In other embodiments, no cable is included, all datacommunication is wireless, and the detector/reading unit 703 and opticaldevices 710 are battery powered.

In an embodiment, the optical devices 710 are configured to continuouslyscan the code 707 information to generate the location information inreal time which is used to generate real time videos or threedimensional images by merging or fusing the two dimensional images whichare simultaneously captured by the endoscopic device 709. In anotherembodiment, the optical devices 710 are configured to scan the code 707information whenever a physician provides instruction to capture anyimage for stamping/tagging that specific image with its locationinformation.

Three-Dimensional and Video Imaging

In another embodiment, the system generates a three dimensional imagebased on user instruction. In an embodiment, the physician firstcaptures a first two dimensional image. Subsequently, the physicianchanges the position of the endoscopic device and captures a second twodimensional image. The physician then provides an instruction to thesystem, using a voice command or a button on the control handle, and thesystem recognizes that the first and the second two dimensional imagescorrespond to a single 3D capture and applies image processingalgorithms to merge the two images to generate a 3D image. In theprocess of merging a plurality of two dimensional images to create athree dimensional image or video, the system performs the steps ofaligning the multiple images to a common rotational angle when therotational angle is different between the different image captures. Thesystem also aligns the brightness, contrast and chromaticity among theimages to bring them to a standard platform.

For example, in one embodiment, a 3D image is captured on a side cameraof an endoscope. The distal tip of the endoscope is straightened topoint forwards as much as possible such that a pathological structure ofinterest lies within the side camera's FOV (Field Of View). A first 2Dimage is captured by the side camera. Tags for the first 2D imagecapture include a depth=81.4 cm and an angle=10.3°. A second 2D image isthen captured by the same side camera after the endoscope has beenmoved. Tags for the second 2D image capture include a depth=82.1 cm andan angle=12.9°. The two 2D images are processed to obtain similarbrightness and color. The first 2D image is shifted up in a number oflines respective with (12.9−10.3)/2=1.3°. This is related with the opticsystem's field of view. The second 2D image is shifted down the samenumber of lines.

The two images are now similar and aligned except they are taken atdifferent endoscope depths. Knowing that the camera type is a sidecamera, the difference in depth is equivalent to the distance betweentwo cameras placed on a same horizontal axis capturing a 3D image. Thisis similar to the way humans capture a 3D image using both eyes.

A software or firmware module constructs a single 3D image from the two2D images in a way similar to that of the human brain. Distance of anobject, or picture element, from the “cameras” is calculated by thedifference in its horizontal positions in the two 2D images. The greaterthe difference between the two images, the shorter the distance.Similarly, a smaller difference means a greater distance. The equivalenthorizontal distance between the positions of the side camera when thefirst and second 2D images are captured is known to the system. In thisexample, it is 82.1 cm−81.4 cm=0.7 cm.

In another example in accordance with embodiments of the presentspecification, a 3D image is captured on a front camera of an endoscope.The distal tip is straightened to point forwards as much as possiblesuch that a pathological structure of interest lies within the frontcamera's FOV (Field Of View). A first 2D image is captured by the frontcamera. Tags for the first 2D image capture include a depth=92.5 cm andan angle=12.3°. A second 2D image is then captured by the front cameraafter the endoscope has been moved. Tags for the second 2D image captureinclude a depth=91.7 cm and an angle=13.4°. The two 2D images areprocessed to obtain similar brightness and color. The first 2D image isrotated clockwise (13.4−12.3)/2=0.55°. The second 2D image is rotatedcounterclockwise 0.55°. The two images are now similar and alignedexcept they are taken at different endoscope depths.

A software or firmware module constructs a single 3D image from the two2D images. Distance of an object, or picture element, from the camera iscalculated by the difference in its size (diameter/circumference/area)in the two 2D images. The greater the difference between the two images,the shorter the distance. Similarly, a smaller difference means agreater distance. The equivalent distance between the positions of thefront camera when the first and second 2D images are captured is knownto the system. In this example, it is 92.5 cm−91.7 cm=0.8 cm.

In embodiments, various parameters, such as the type of coding schemeand the spacing between codes, influence the selection of type andnumber of optical devices required in the detector/reading unit, such asdetector/reading unit 703 of FIG. 7. In embodiments, the selection isalso based on the primary requirement that, at all times, the field ofview of at least one optical device should encompass a code or any othertype of marking on the device.

In various embodiments, the length of the detector/reading unit 703varies between 5-12 cm and the inner diameter of the detector/readingunit 703 varies between 5-8 cm. One of ordinary skill in the art wouldappreciate that the above ranges are mentioned just for exemplarypurpose and one can configure a reader unit with other suitabledimensions without departing from the spirit and scope of thisspecification.

The use of a long and narrow device such as detector/reading unit 703also prevents the random spray of fecal matter in all directions in casethe patient suddenly has a bowel movement. Such matter shall becontained within the cylindrical structure or may be sprayed as a narrowcone. With this feature, physicians can avoid using accessories such asmasks and goggles. It is more convenient for the physicians to conductprocedures without such accessories and also the image quality seen byphysicians will be better in case they are not wearing goggles.

In an embodiment, the underwear 701 is designed such that it can be usedas a standalone piece of equipment which the patients can wear toprotect their modesty even if the location detection is not performedand the corresponding location detector/reading unit 703 is not attachedto the patient.

Conical Detector/Reading Unit

FIG. 8 illustrates an alternate embodiment of a detector/reading unit800 in the form of a truncated hollow conical structure. A hollowportion or opening 805 in the center of the truncated cone allows thephysician to insert medical tools therethrough and into a patient's bodylumen. The inside surface 806 of the conical detector/reading unit 800comprises a plurality of optical devices to read codes marked on medicaltools, such as insertion tube 801 of an endoscopic device, forestablishing the location information of such a device. In embodiments,the detector/reading unit 800 is structured such that during aprocedure, the radius of its proximal end 802 positioned towards thephysician is greater than the radius of its distal end 803 positionedtowards the patient. In various embodiments, the inner and outerdiameters of the detector/reading unit 800 at its distal end 803 are inranges of 3-5 cm and 4-6 cm respectively. In various embodiments, theinner and outer diameters of the detector/reading unit at its proximalend 802 are in ranges of 5-7 cm and 6-8 cm respectively. Such a conicalstructure includes greater clearance 804 at the proximal end 802 whichprovides increased maneuverability for the physician while operating theendoscopic device as compared to operating with a detector/reading unithaving a cylindrical structure of uniform radius. The conical structureis further superior to the cylindrical structure as it minimizesfriction between the insertion tube 801 of the endoscope and thedetector/reading unit 800 which could wear out the endoscope's outerskin or markings. Further, a conical unit is more convenient forphysicians as the potential friction between the endoscope and acylindrical detector/reading unit may strain the physician, requiringhim to apply extra force for longitudinal or rotational motion.

Ring-Shaped Detector/Reading Unit

FIG. 9 illustrates an alternate embodiment of a detector/reading unit900 comprising a ring shaped base unit 901. As shown in FIG. 9, thedetector/reading unit 900 comprises a ring shaped base unit 901 which isadapted to firmly attach or couple to the body of a patient eitherdirectly or with the assistance of a garment, such as underwear,designed for this purpose and described in this specification. A set ofarms 904 extend proximally from the base unit 901 and comprise thereading devices, such as optical devices, embedded therein. When in use,the base unit 901, at the distal end 903 of the detector/reading unit900, is positioned toward the patient and the arms 904, at the proximalend 902 of the detector/reading unit 900, are positioned toward thephysician. The configuration described in FIG. 9 offers the benefit oflighter weight as compared to the cylindrical shaped and conical shapeddetector/reading units described in FIG. 7 and FIG. 8 respectively,while the cylindrical and conical shaped detector/reading units offerhigher structural stability.

Pathological Structure Size Determination

In an embodiment of the present specification, the size of apathological structure, such as a polyp or any other abnormality, can bemeasured using the location information corresponding to variouscaptured images of the said pathological structure. Referring to FIG.10, a mathematical method to calculate the size is described wherein:

S 1004=Actual size of Pathological Structure 1002;

D 1006=Distance to the pathological structure 1002 in a first image;

A 1008=Difference is insertion depth between two consecutive images(obtained from a detector/reading unit);

D 1006+Δ 1008=Distance to the pathological structure 1002 in a secondimage;

Pxl1=the size in pixels of pathological structure 1002 when theendoscope's distance is D 1006;

Pxl2=the size in pixels of the pathological structure 1002, when theendoscope's distance is D 1006+Δ 1008;

tg=tangent;

tg⁻¹=arctangent; and

K=a factor translating from angular field of view to number of pixels.

Then:

${{{Pxl}\; 1} = {2{Ktg}^{- 1}\frac{S}{2D}}};$${{Pxl}\; 2} = {2{Ktg}^{- 1}\frac{S}{2\left( {D + \Delta} \right)}}$

And so both D and S can now be extracted:

$D = {\Delta\frac{{tg}\left( {{Pxl}\;{2/2}K} \right)}{{{tg}\left( {{Px}\; l\;{1/2}\; K} \right)} - {{tg}\left( {{Pxl}\;{2/2}K} \right)}}}$S = 2Dtg(pxl 1/2K)

Therefore S, the size of a pathological structure, is a function of: theendoscope's distance D, the pixel size of the structure at distance D,and the factor K. As described above, the size S 1004 of thepathological structure 1002 is deduced from the size in pixels of saidpathological structure 1002 captured in two different image shots atinsertions whose depth difference is known.

Real Time Image Mapping in Endoscopic Procedures

FIG. 11 illustrates a physician and patient and an endoscopy system inaccordance with one embodiment of the present specification. The figureschematically depicts a layout of an endoscopy system 1110 and anassociated interface unit 1100 deployed in an operation room, accordingto some embodiments. A patient 1180 is supported on a bed 1182 and aphysician 1184 employs an endoscope 1120 of endoscopy system 1110 duringa medical procedure.

The endoscope 1120 is connected to a main controller 1130 by a utilitycable 1132. In embodiments, the endoscope 1120 provides threesimultaneous endoscopic views using three viewing elements housed in adistal tip 1150 of endoscope 1120. In embodiments, the main controller1130 is connected to three display screens, 1140 a, 1140 b, and 1140 c.In an embodiment, each display screen is configured to display acorresponding view of the three endoscopic views provided by endoscopysystem 1110, as described above. In an embodiment, the main controller1130 is connected to at least one display configured to display acorresponding view of the three endoscopic views provided by endoscopysystem 1110.

In embodiments, a first sensor (not shown) is mounted in distal tip 1150of the endoscope 1120. In an embodiment, the first sensor comprises atransmitter/transceiver and a second sensor or base unit is located onor proximal to endoscope 1120 such that it remains outside the bodycavity where endoscope 1120 is inserted. In embodiments, the secondsensor is placed on the body of the patient 1180 such as at location1190 on the surface of patient's 1180 body. Location 1190 may be alocation near an opening on the patient 1180 body from where endoscope1120 is inserted, such as the patient's rectum. In another embodiment,the second sensor 1190 is placed at a pre-defined location on a bed 1182of the patient 1180 undergoing the endoscopic scan. In this case, thepatient 1180 may be stably secured to the bed 1182. In anotherembodiment, the second sensor 1190 is placed at a pre-defined locationon a flexible tube of endoscope 1120 between its handle and the body ofthe patient 1180. In embodiments, the second sensor 1190 includes areceiver/transceiver that communicates with the first sensor.

In an embodiment, the first sensor senses a location of the endoscopetip within the lumen and transmits the same in real time to the secondsensor 1190. In embodiments, the first sensor captures image informationin three dimensions, and also records the time of image capture. Thefirst sensor transmits image, time, location and any other informationto second sensor 1190 by using wireless signals lying in the sub-Gigahertz frequency field. In an embodiment, both sensors use frequenciesbelow 1 gigahertz and within ranges such as 30-1000 MHz, 30-150 MHz,150-1000 MHz, or any other range authorized for medical uses. Thereceived endoscope tip location information is processed using acontinuous signal stream to provide a map of the scanned portions of thelumen in real time. In an embodiment, the map is displayed on one ormore display screens coupled with endoscope 1120. In variousembodiments, both sensors are of any suitable type, such asaccelerometers, gyro devices and radar.

Also, in an embodiment, the first sensor functions as a navigation unitthat sends real time endoscope-tip navigation information to secondsensor 1190 that functions as a base unit. In an embodiment, the secondsensor 1190 provides a reference plane for obtaining a real timeposition of the distal tip of endoscope 1120. A pre-defined algorithmmay be used to process the reference plane obtained from sensor 1190 andthe navigation co-ordinates obtained from the first sensor to provide alocation of the endoscope tip in real time. In embodiments, thereal-time location information is provided within a rectangular range of0.1-3 mm and 0.1-1 mm. The rectangular accuracy of the obtained positiondepends upon a frequency of navigation co-ordinates received from thefirst sensor.

In an embodiment, sensor 1190 is only a transceiver to provide areference plane for obtaining a real time position of the distal tip ofendoscope 1120. In another embodiment, the first sensor is a radar basedsensor for capturing a current location of the endoscope tip andtransmitting this information to sensor 1190.

FIG. 12 is a flowchart illustrating a method of obtaining and using areal time image map of an endoscope tip from within a body cavity duringan endoscopic procedure, in accordance with an embodiment of the presentspecification.

At step 1202, an operating physician commences an endoscopic scan bybeginning a recording session. In an embodiment, a time synch operationis also performed at the commencement of the endoscopic scan. In anembodiment, a recording session is started by pressing a button providedon a handle portion of the endoscope immediately before inserting theendoscope tip into a body cavity. In other embodiments, the recordingsession is started by operating any pre-defined control. In variousembodiments, once the recording session begins, internal images of thescanned body cavity captured by the imaging elements positioned in theendoscope tip are recorded in a predefined format in a specifieddatabase.

At step 1204, an initial position of the endoscope tip within the bodycavity being scanned is recorded as zero coordinates. At step 1206, areal time map of the images captured by the endoscope tip travellingwithin the body cavity is displayed on one or more display devicescoupled with the endoscope. The image map is drawn by using the zerocoordinates as an initial reference point. U.S. patent application Ser.No. 14/697,933, entitled “System and Method of Scanning a Body CavityUsing a Multiple Viewing Elements Endoscope” and filed on Apr. 28, 2015,describes one method for mapping images of a body cavity captured by anendoscope in real time onto a reference image and is herein incorporatedby reference in its entirety. U.S. patent application Ser. Nos.14/505,387 and 14/505,389, both entitled “Endoscope with IntegratedSensors” and filed on Oct. 2, 2014, are also herein incorporated byreference in their entirety.

At step 1208, the real time endoscopic image map is recorded in adatabase. At step 1210, the recorded real time endoscopic image map ofthe body cavity is used to perform analytical operations using data fromone or more previous endoscopic scans and a pre-defined algorithm. In anembodiment, a pace of the endoscope tip scanning the lumen is alsodisplayed and recorded by using information from steps 1206-1208 tocalculate a difference between two subsequent endoscope tip positions.

At step 1212, the operating physician marks one or more regions ofinterest on the endoscopic image of the body cavity being displayed inreal time on one or more display devices, the regions of interest beingrelated to observation of an abnormality. In an embodiment, theoperating physician also records one or more comments related to theobserved abnormality with respect to the marked regions. In anembodiment, a plurality of formats and colors are made available formarking the regions of interest and recording corresponding remarks. Inan embodiment, an operating physician also records audio commentscorresponding to a marked region.

At step 1214, if during the endoscopic scan, the endoscope tip arrivesat a particular location within the body cavity which was marked duringan endoscopic scan performed at an earlier time, an audio and/or visualalert is displayed. In various embodiments, the alert is generated andtriggered when the sensor tip location is recorded and, in real-time,compared against saved marked locations from prior scans wherein saidsaved locations are associated with a region of interest. A user canload the previous record and display it synchronously with a liveprocedure. This is achieved with the time and position informationcollected by the first sensor at the distal tip of the endoscope. Theuser or operating physician is alerted and he may consult the endoscopicmap of the same body cavity captured and recorded at an earlier time inorder to compare and/or assess the condition of an observed abnormality.In an embodiment, the endoscopic image map captured at an earlier timeis displayed in a portion of the display screen displaying the real timeimage map.

In an embodiment, the endoscopic image map captured at an earlier timeis displayed at the same pace at which the endoscope tip is scanning thelumen at a current time. Hence, the speed of a video displaying anendoscopic image map captured at an earlier time (dependent upon thespeed at which the endoscope was traversing a body lumen) is adjusted tobe in synch with the speed at which the operating physician moves theendoscope distal tip within the lumen in the current procedure, enablingthe physician to compare the images captured at different times. In anembodiment, assuming an insertion depth X in a current endoscopyprocedure, the displayed reference image frame will be the closest onerelated to depth X. This is to allow the physician to compare identicalsections of the lumen between two procedures separated in time(reference and current). An insertion speed at point X in the referenceendoscopic procedure is defined as Vr and a current procedure insertionspeed at the same point X is defined as Vc. When Vc<Vr, the rate ofpulling frames from the reference will be slower than the frame ratesent to display screens. This means some of reference frames would beduplicated when sent to the display screens. When Vc>Vr, the rate ofpulling frames from the reference will be faster than the frame ratesent to display screens. This means some of the reference frames wouldbe skipped when sent to the display screens.

At step 1216, an extraction of the endoscope tip from within the bodycavity signals an end of the recording session. In an embodiment, theoperating physician is required to operate a control (such as pressing abutton provided on the endoscope) in order to stop the recordingsession, whereas in another embodiment, the extraction of endoscope tipfrom within the body cavity automatically stops the recording session.In various embodiments, automatic cessation of recording occurs as soonas the detector/reading unit can no longer detect markings on theinsertion tube. In other words, recording automatically stops once theinsertion tube has been removed from the reader such that there are nomarkings for the detector/reading unit to detect. The operatingphysician can re-insert the endoscope tip into the body cavity to begina new recording session. Once a recording session ends, an updatedendoscopic image map is displayed on one or more display devices coupledwith the endoscope.

In an embodiment, the present specification provides a system and methodfor transmitting images and videos of endoscopic procedures to aplurality of devices by using wireless network technology. In anembodiment, a main control unit of the endoscope is provided with awireless module that establishes a link over wireless platforms such asIEEE 802.11ac or IEEE 802.11ad which are used for transmitting livevideo of endoscopic procedures in formats such as 1080p at a frequencyof 59.94 Hz. In an embodiment, live videos of endoscopic procedures aredisplayed on one or more display devices connected via the wireless linkto the endoscope's main control unit.

In an embodiment, one or more portable devices such as but not limitedto cell phones, tablets, laptops, and other illustrative devices such asthree dimensional demonstration, 3D holography, present in the vicinityof the endoscope being used to perform a patient's scan are used toperform functions such as live broadcast of the endoscopic scan,database searches to obtain a required patient's scan results andcompare them with one or more scan results. Further, in an embodiment, amobile application may be provided to be used in conjunction with awireless module installed in a main control unit of the endoscope andone or more mobile devices to enable the operating staff, such asnurses, to obtain all of a patient's endoscopic scan data stored in apredefined database along with a live video of a current on-going scan.In an embodiment, the mobile application provides functionalities suchas zooming in and scrolling through the endoscopic scan data displayedon a mobile device.

In an embodiment, a wireless module installed within a main control unitof the endoscope is connected to a secured Internet service which isused to obtain software/firmware updates for the main control unit. Inan embodiment, the software/firmware updates are obtained via a pushdata method wherein the updates can be downloaded and installedautomatically whenever the main control unit is connected to the securedInternet link, or be programmed to check for updates at pre-definedintervals. In an embodiment, a message indicating an available update isdisplayed on a screen of the main control unit, enabling a user tochoose to proceed with the update. Also, in various embodiments, astatus of the main control unit does not change during an endoscopicprocedure. Any updates/changes to software/firmware of the main controlunit take place only when the endoscope is not being used for scanning apatient body.

In an embodiment, the wireless module and the secured Internet link isalso used for performing maintenance activities on the control unit bytransmitting required diagnostic applications and software/firmwareupdates, without having to send support personnel to the endoscope' slocation.

In an embodiment, a wireless module installed within a main control unitof the endoscope connected to a secured internet service is used by anoperating physician to consult in real time with one or more doctorspresent at separate geographical locations regarding an ongoingendoscopic procedure via audio/textual methods. A video coverage of theprocedure is transmitted to one or more doctors at remote locations inreal time along with the operating physician's queries. This feature ofthe present specification enables an operating physician to take helpfrom a plurality of doctors present at remote locations during anendoscopic procedure for improving the diagnostic quality of theprocedure.

In an embodiment, a recording of an endoscopic procedure is made in realtime by using the wireless module installed within the main controlunit. The recording serves as a back-up record of the procedure and isstored in one or more databases. The recorded procedures stored indatabases can be used at a later date for patient treatment as well asfor education and training purposes. In an embodiment, every recordedprocedure is synchronized to a pre-defined clock time by a qualifiedprofessional.

In an embodiment, in addition to a wireless module installed within themain control unit, one or more Wi-Fi modules are also installed in otherparts of an endoscope, such as the endoscope handle. In such a scenariowhere multiple Wi-Fi modules are installed at multiple positions on theendoscope, the precise location of the endoscope tip within a patient'sbody is obtained by correlating the location information received viaeach Wi-Fi module.

Video Processing Architecture

FIG. 13 is a block diagram illustrating an exemplary video processingarchitecture, according to an embodiment of the present specification.FIG. 13 details how a video controller 1320 containing a controllercircuit board (Base Board Module) 1352 of a main control unit (1130 ofFIG. 11) operatively connects with an endoscope 1310 and display units1350. Referring to FIG. 13, the video controller 1320 (which containscontroller circuit board 1352) comprises a camera board 1321 thatcontrols the power supplies to LEDs 1311, transmits controls for theoperation of image sensor(s) 1312 (comprising one or more cameras) inthe endoscope, and converts pre-video signals from image sensors tostandard video signals. The image sensor(s) 1312 may be a charge coupleddevice (CCD) or a complementary metal oxide semiconductor (CMOS) imager.The camera board 1321 receives pre-video signal(s) 1313 generated by theCCD imager and also other remote commands 1314 from endoscope 1310. Inone embodiment, PC 1326 is connected directly to FPGA 1323 and notthrough MPEG 1322, which is an optional component.

Video Controller 1320 further comprises elements for processing thevideo obtained from the image sensors 1312 through camera board 1321, aswell as other elements for system monitoring and control.

All these elements are connected with a Base Board Module 1352, which isa printed circuit board (PCB). In an embodiment, elements such as ICs(Integrated Circuits) are connected by soldering, an element 1326 (SOMor System on Module) is connected by mounting, while all other elementsare connected by means of cables.

In some embodiments, receiving the optical information (line coding, barcodes) or RFID, as described in various embodiments of the presentspecification, is performed through a USB connection with the maincontrol unit (1130 of FIG. 11). In various embodiments, processing ofthe positional information is performed jointly by the FPGA 1323, PCsystem on module 1326, and a GPU of the main control unit. In oneembodiment, a plurality of codes on the insertion tube of an endoscopicdevice comprises numbers representing the insertion depth at anyposition over the length of the insertion tube. The detector/readingunit contains one or more optical devices (i.e. cameras) for detectingthe printed numbers. The video carrying the images of said printednumbers is processed by the PC system on module 1326, FPGA 1323, andsystem GPU, by means of computerized vision, to establish the depth ofinsertion. Display information regarding the position of the endoscopeis displayed similarly to any other on screen display (OSD) or graphicoverlay type.

Various elements on Base Board Module 1352 are described as follows:

FPGA (Field Programmable Gate Array) 1323:

FPGA 1323 is a logic device programmed specifically for the systemrequirements and performs tasks that may be categorized by two types:logic tasks which are implemented by hardware (i.e. should not or mustnot be implemented by software), and logic tasks related to video imageprocessing. In an embodiment, the Base Board Module 1352 includes one ormore double data rate type three synchronous dynamic random accessmemory modules (DDR3) 1328 in communication with FPGA 1323.

Among logic tasks related to video image processing, FPGA 1323 generatesan internal test pattern that may be provided to video outputs via aVideo Output Interface 1324 to multiple display units 1350. Inembodiments, a wireless video transmitter 1380 is connected to the FPGA1323 and the video output interface 1324. In an embodiment, thetransmitter 1380 is configured to wirelessly transmit video output towireless receivers among display units 1350, through a wireless antenna1382. In various embodiments, the Base Board Module 1352 includes one ormore wireless video transmitters 1380. For example, in some embodiments,wherein the endoscope 1310 is a gastroscope, the Base Board Module 1352includes one or two wireless video transmitters 1380. In anotherembodiment, wherein the endoscope is a colonoscope, the Base BoardModule 1352 includes three wireless video transmitters 1380. In anotherembodiment, the endoscope comprises P cameras, M transmitters, and Nscreens, where each of M, N, and P are different natural numbers. Wherethere are additional transmitters than cameras, the additionaltransmitters may transmit placeholder, screen saver, or other genericdisplay information.

In an embodiment, a wireless transmitter 1380 installed within the maincontrol unit of the endoscope is connected to a secured Internet servicewhich may be used to obtain software/firmware updates for the maincontrol unit. In an embodiment, the software/firmware updates areobtained via a push data method wherein the updates can be downloadedand installed automatically whenever the main control unit is connectedto the secured Internet link, or be programmed to check for updates atpre-defined intervals. In an embodiment, a message indicating anavailable update is displayed on a screen of the main control unit,enabling a user to choose to proceed with the update. Also, in variousembodiments, a status of the main control unit does not change during anendoscopic procedure. Any updates/changes to software/firmware of themain control unit take place only when the endoscope is not being usedfor scanning a patient body.

In an embodiment, the wireless module, including transmitter 1380 andthe secured Internet link may also be used for performing maintenanceactivities on the control unit by transmitting required diagnosticapplications and software/firmware updates, without having to sendsupport personnel to the endoscope's location.

In an embodiment, the wireless module installed within a main controlunit of the endoscope connected to a secured internet service may beused by an operating physician to consult in real time with one or moredoctors present at separate geographical locations regarding an ongoingendoscopic procedure via audio/textual methods. A video coverage of theprocedure may be transmitted to one or more doctors at remote locationsin real time along with the operating physician's queries. This featureof the present specification enables an operating physician to take helpfrom a plurality of doctors present at remote locations during anendoscopic procedure for improving the diagnostic quality of theprocedure.

In an embodiment, a recording of an endoscopic procedure is done in realtime by using the wireless module installed within the main controlunit. The recording may serve as a back-up record of the procedure andmay also be stored in one or more databases. The recorded proceduresstored in databases may be used at a later date for patient treatment aswell as for education and training purposes. In an embodiment, everyrecorded procedure is synchronized to a clock time pre-defined by aqualified professional.

In an embodiment, in addition to a wireless module installed within themain control unit, one or more Wi-Fi modules are also installed in otherparts of an endoscope, such as the endoscope handle. In such a scenariowhere multiple Wi-Fi modules are installed at multiple positions on theendoscope, the precise location of the endoscope tip within a patient'sbody is obtained by correlating the location information received viaeach Wi-Fi module.

DSP (Digital Signal Processor) 1322:

DSP 1322 is used for recording compressed (coded) video and playing backdecompressed (decoded) video. In an embodiment, the standard ofcompressed video is H264 or equivalent (such as MPEG). In anotherembodiment, the video controller 1320 does not include a digital signalprocessor and the PC system on module 1326 communicates directly withthe FPGA 1323.

Operationally, the FPGA 1323 selects for DSP 1322, the desired video tobe recorded, i.e. any of the inputs, or, more likely, a copy of one ormore of the screens. In the latter case, this includes the OSD andformat conversion. In the likely case of the screen's format differingfrom that of DSP's 1322 required video input format, the FPGA 1323 alsoconverts the screen's format to the desired DSP 1322 format whiletransmitting video to DSP 1322.

Auxiliary Video Input Interface 1325:

In an embodiment, the video input to the Auxiliary Video Input Interface1325 comprises analog video, such as in CVBS (color, video, blanking,sync), S-Video or YPBPR format or digital video (DVI), and may bedisplayed as such.

SOM (System on Module) 1326:

SOM 1326 provides an interface to input devices such as keyboard, mouse,and touchscreen via a Touch I/F. Through these input devices, togetherwith the buttons in a Front Panel, the user controls the system'sfunctionality and operational parameters. In an embodiment, a peripheralcomponent interconnect express (PCIe) bus connects SOM 1326 with FPGA1323. Most common types of data traffic over the PCIe may include:

a. SOM 1326 to FPGA 1323: Commands (for example, when the user changesoperational parameters); andb. FPGA 1323 to SOM 1326: Registers values, which provide an indicationof the internal status, and captured images.

Other Functionalities:

Video Controller 1320 further controls one or more fluid, liquid and/orsuction pump(s) which supply corresponding functionalities to theendoscope through a pneumatic I/F, pump and check valve. VideoController 1320 further comprises an on-board power supply 1345 and afront panel which provides operational buttons for the user.

Camera board 1321 receives video signal 1313 which, in an embodiment,comprises three video feeds, corresponding to video pickups by threeendoscopic tip viewing elements (one front and two side-looking viewingelements), as generated by image sensors 1312. In an embodiment, thethree video feed pickups, corresponding to the three viewing elements(the front-looking, left-side looking and right-side looking viewingelements) of an endoscopic tip (such as the three viewing elements ofthe tip section 150 of FIG. 1B), are displayed on three respectivemonitors.

In an embodiment, display units 1350 includes one or more portabledevices, such as but not limited to, cell phones, tablets, laptops, andother illustrative devices such as three dimensions demonstration, 3Dholography, present in the vicinity of the endoscope being used toperform a patient's scan. Display devices, such as those includingwireless receivers/transceivers, are used to perform functions such aslive broadcast of the endoscopic scan, database searches to obtain arequired patient's scan results and compare them with one or more scanresults, etc.

Further, in an embodiment, a mobile application is provided to be usedin conjunction with a wireless module installed in a main control unitof the endoscope and one or more mobile devices to enable operatingstaff, such as nurses, to obtain all of a patient's endoscopic scan datastored in a predefined database along with a live video of a currenton-going scan. The mobile application provides functionalities such aszooming in and scrolling through the endoscopic scan data displayed on amobile device.

The above examples are merely illustrative of the many applications ofthe system of present invention. Although only a few embodiments of thepresent invention have been described herein, it should be understoodthat the present invention might be embodied in many other specificforms without departing from the spirit or scope of the invention.Therefore, the present examples and embodiments are to be considered asillustrative and not restrictive, and the invention may be modifiedwithin the scope of the appended claims.

1-23. (canceled)
 24. An endoscope assembly comprising: an endoscopecomprising: a control handle configured to control one or more functionsof the endoscope; an insertion tube having a length extending distallyfrom said control handle; and at least one imaging assembly at a tipsection of the insertion tube configured to generate image data of aplurality of areas in a field of view of the at least one imagingassembly, and to transmit the image data to a processor; a displayscreen; and a main control unit coupled to said control handle andcomprising the processor; wherein the display screen is configured todisplay the image data in a first portion of the display screen anddisplay an endoscopic image map captured prior to generation of theimage data in a second portion of the display screen.
 25. The endoscopeassembly of claim 24, wherein the insertion tube includes a plurality ofmarkings, and the endoscope assembly further comprises: a detectorconfigured to detect said plurality of markings during insertion of theinsertion tube into a subject, and to transmit data representative ofsaid plurality of markings to the processor.
 26. The endoscope assemblyof claim 25, wherein the main control unit is adapted to merge aplurality of image frames of the image data, based on the transmitteddata representative of the plurality of markings, to generatethree-dimensional image data of the areas viewed by the at least oneimaging assembly.
 27. The endoscope assembly of claim 25, wherein thedetector is adapted to surround a portion of the length of the insertiontube and remain outside of the control handle.
 28. The endoscopeassembly of claim 25, wherein the detector is configured to detectrotation of the insertion tube about a central longitudinal axis of theinsertion tube.
 29. The endoscope assembly of claim 25, wherein theprocessor is adapted to use the data representative of the plurality ofmarkings to calculate location information related to the location ofthe endoscope, wherein the image data includes a plurality of imageframes, and wherein the processor is adapted to stamp each image framewith the location information.
 30. The endoscope assembly of claim 28,wherein the image data includes a first image frame and a second imageframe; wherein the processor is adapted to store a rotation angleassociated with each of the first image frame and the second imageframe; wherein the first image frame has a rotation angle different fromthe rotation angle of the second image frame; and wherein the processoris configured to align the first image frame and the second image frameto a common rotation angle.
 31. The endoscope assembly of claim 25,wherein the processor is adapted to determine a speed at which theinsertion tube is moving through a patient's body using the datarepresentative of said plurality of markings; and wherein the displayscreen is configured to display the speed at which the insertion tube ismoving through the patient's body in a third portion of the displayscreen.
 32. The endoscope assembly of claim 31, wherein the control unitis adapted to generate a notification in the form of (i) a signal to avibrator in the control handle, (ii) a signal to an audio speaker, or(iii) a signal to the display screen to visually present an icon, image,color, or other picture when the speed at which the insertion tube ismoving through the patient's body exceeds a threshold speed value. 33.An endoscope assembly comprising: an endoscope comprising: a controlhandle configured to control one or more functions of the endoscope; aninsertion tube having a length extending distally from said controlhandle and including a plurality of markings; and at least one imagingassembly at a tip section of the insertion tube configured to generateimage data of a plurality of areas in a field of view of the at leastone imaging assembly, and to transmit the image data to a processor; adisplay screen; a detector configured to detect the plurality ofmarkings during insertion of the insertion tube into a subject, and totransmit data representative of said plurality of markings to theprocessor; and a main control unit coupled to said control handle andcomprising the processor; wherein the processor is adapted to determinea speed at which the insertion tube is moving through the subject usingthe data representative of the plurality of markings; and wherein thedisplay screen is configured to display the speed at which the insertiontube is moving through the subject in a portion of the display screen.34. The endoscope assembly of claim 33, wherein the main control unit isadapted to merge the image data with the data representative of theplurality of markings to generate three-dimensional image data of theareas viewed by the at least one imaging assembly.
 35. The endoscopeassembly of claim 33, wherein the imaging assembly includes a pluralityof imaging assemblies.
 36. The endoscope assembly of claim 33, whereinthe detector is adapted to surround a portion of the length of theinsertion tube and remain outside of the control handle.
 37. Theendoscope assembly of claim 33, wherein the detector is configured todetect rotation of the insertion tube about a central longitudinal axisof the insertion tube.
 38. The endoscope assembly of claim 33, whereinthe processor is adapted to use the data representative of the pluralityof markings to calculate location information related to the location ofthe endoscope.
 39. The endoscope assembly of claim 33, wherein the imagedata includes a plurality of image frames, wherein the processor isadapted to map the plurality of image frames in a real time image map;and wherein the display screen is configured to display the real timeimage map in a first portion of the display screen and display anendoscopic image map captured prior to generation of the image data in asecond portion of the display screen.
 40. The endoscope assembly ofclaim 33, wherein the processor is adapted to merge a plurality of imageframes of the image data, based on the transmitted data representativeof the plurality of markings, to generate three-dimensional image dataof the areas viewed by the imaging assembly.
 41. An endoscope assemblycomprising: an endoscope comprising: a control handle configured tocontrol one or more functions of the endoscope; an insertion tube havinga length extending distally from said control handle; and at least oneimaging assembly at a tip section of the insertion tube configured togenerate image data of a plurality of areas in a field of view of the atleast one imaging assembly, and to transmit the image data to aprocessor; a display screen; and a main control unit coupled to saidcontrol handle and comprising the processor; wherein the display screenis configured to display an endoscopic image map captured prior togeneration of the image data in synch with a current speed at which theinsertion tube moves within a subject.
 42. The endoscope assembly ofclaim 41, wherein the insertion tube includes a plurality of markings,and the endoscope assembly further comprises: a detector configured todetect said plurality of markings during insertion of the insertion tubeinto a subject, and to transmit data representative of said plurality ofmarkings to the processor; wherein the processor is adapted to determinethe current speed using the data representative of said plurality ofmarkings.
 43. The endoscope assembly of claim 41, wherein the at leastone imaging assembly includes a plurality of imaging assemblies.